Remote and locally located cameras typically include devices for camera control. Devices include stepping motors or other mechanisms configured to point the camera or an image capturing device toward a scene or point of interest. Examples include teleconferencing applications, surveillance cameras, security cameras, cameras that are remotely controlled, activated by motion, light or other stimuli, remote sensing cameras such as those placed on robotic means (examples including those used in space exploration, deep sea diving, and for sensing areas or scenes to dangerous or inaccessible for normal camera operations (inside nuclear reactor cores, inside pipes, police cars, or law enforcement robotics, for example).
Normally, cameras are manually operated by a human operator on site, or remotely controlling the camera via a steering input (joystick or mouse, for example). In the case of remotely steered cameras, steering inputs generally activate a control program that sends commands to a stepping motor or other control device to steer a camera toward an object, item, or area of interest. General zooming functions of the camera may also be activated either on site or remotely.
In the case of teleconferencing applications (meetings, lectures, etc.), a variable angle camera with a mechanical tilt, pan, focal length, and zoom capability is normally used. Such devices generally require a human operator to orient, zoom, and focus a video or motion picture camera. In some cases, conference participants may be required to activate a specific camera or signal attention of a camera configured to zoom in or focus on selected areas of a conference room.
Multiple cameras have been utilized in a number of applications. For example, Braun et al., U.S. Pat. No. 5,187,571, “TELEVISION SYSTEM FOR DISPLAYING MULTIPLE VIEWS OF A REMOTE LOCATION,” teaches an NTSC camera array arranged to form an aggregate field, and Henley, U.S. Pat. No. 5,657,073, “SEAMLESS MULTI-CAMERA PANORAMIC IMAGING WITH DISTORTION CORRECTION AND A SELECTABLE FIELD OF VIEW,” teaches a system for production of panoramic/panospheric output images.
Applications for multiple or steerable cameras include teleconferencing systems that typically direct a camera toward a speaker who is then broadcast to other teleconference participants. Direction of the camera(s) can be performed manually, or may utilize a tracking mechanism to determine a steering direction. Some known tracking mechanisms include, Wang et al., “A Hybrid Real-Time Face Tracking System,” in Proc. ICASSP '98, and, Chu, “Superdirective Microphone Array for a Set-Top Videoconferencing System,” In Proc. ICASSP '97.
However, technical challenges and costs have prevented such systems from becoming common and in wide spread use.
Systems attempting to integrate multiple images have failed to meet the needs or goals of users. For example, McCutchen, U.S. Pat. No. 5,703,604, “IMMERSIVE DODECAHEDRAL VIDEO VIEWING SYSTEM,” teaches an array of video cameras arrayed in a dodecahedron for a complete spherical field of view. Images are composed at the receiving end by using multiple projectors on a hemispherical or spherical dome. However, the approach taught in McCutchen will suffer problems at image boundaries, as the multiple images will not register perfectly and result in obvious “seams.” Additionally, object disparity may result due to combining overlapping images taken from multiple cameras.
In another example, Henley et al., U.S. Pat. No. 5,657,073, “SEAMLESS MULTI-CAMERA PANORAMIC IMAGING WITH DISTORTION CORRECTION AND SELECTABLE FIELD OF VIEW,” teaches combining images from radially-arranged cameras. However, Henley fails to disclose any but radially-arranged cameras, and does not provide details on image composition methods.
In another example, Nalwa, U.S. Pat. No. 5,745,305 issued on Apr. 28, 1998, entitled “PANORAMIC VIEWING APPARATUS,” teaches a four-sided pyramid-shaped element to reflect images from four different directions to four different cameras. Each camera is positioned to receive a reflected image from one of the reflective sides of the pyramids. The cameras are arranged so that they share a perceived Center Of Projection (“COP”) located at the center of the pyramid. A common COP means there can be no overlap region to blend camera images, so seams will be more apparent.
Panoramic viewing systems using optical means to achieve a common COP are expensive and difficult to manufacture to the necessary degree of precision. Manufacturing requires precision alignment of the pyramid and cameras to generate the perceived COP. If one of the cameras is initially misaligned, subsequently moved or the pyramid element is shifted, distortion may occur between combined images.
Accordingly, there is a desire to provide a cheaper, more reliable, and faster multiple camera viewing system for generating panoramic images.